Data communication networks may include various switches, routers, hubs, and other devices coupled to and configured to receive data and forward the data on the network. These devices will be referred to herein as “network elements.” A network element is generally not a consumer of the data, but rather is used to receive and forward data so that the data may pass through the network. Data is communicated through a network by enabling the network elements to pass protocol data units, such as frames, packets, cells or segments, between each other over communication links. A particular protocol data unit may be handled by multiple network elements and cross multiple communication links as it travels between its source and its destination over the network.
The various network elements on the communication network communicate with each other using predefined sets of rules, referred to herein as protocols. Different protocols are used to govern different aspects of the communication, such as how signals should be formed for transmission between network elements, various aspects of what the protocol data units should look like, how protocol data units should be handled or routed through the network by the network elements, and how information such as routing information should be exchanged between the network elements.
In optical networks such as SONET networks, a connection will be established before data is allowed to be passed over the optical network. The connections are provisioned through the network so that a particular service instance will be connected between ports on network elements and remain provisioned over those ports until the provisioning is changed or shut down. For example, once the optical network elements are interconnected by optical fibers, a network manager will log into an optical service provisioning system to set up data connections (services) through the network elements and optical fibers.
Optical services are offered in blocks of bandwidth of a particular size (channels). For example, on a SONET network an STS-1 provides 50 Mb of bandwidth, an STS-3c provides 150 Mb of bandwidth, and an STS-12c provides 600 Mb of bandwidth. Where Gigabit Ethernet is to be transported over a SONET network, the Gigabit Ethernet service is too large to be carried on an STS-12c. Unfortunately, the next larger size (STS-48c) offers 2.5 Gb, which is much larger than necessary to carry the 1 Gigabit Ethernet service. Similar channel sizes are offered in SDH networks prevalent in regions outside of North America.
As Ethernet services become more prevalent, the need to accommodate transmission channels of different sizes caused an extension to the SONET standard to be adopted which is commonly referred to as Virtual Concatenation, or VCAT for short. VCAT enables multiple smaller channels (i.e. STS-1s or STS-3c's) to be grouped together logically to form a larger channel through the SONET network. Keeping with the Gigabit Ethernet example, 21 STS-1s may be concatenated together to create a channel through the optical network with approximately 1 Gb of bandwidth. Alternatively, 7 STS-3c's may be concatenated together to create the same sized channel. One restriction imposed by VCAT is that all channels that are grouped together must be of the same size. Thus, for example, it is not possible to concatenate a STS-12 with a STS-3.
When multiple STS-1s or STS-3c's are concatenated together, the STS-1s or STS-3c's channels are actually separate time slots that may be reserved on the same or different physical fibers of the network. In operation, the data that is to be carried on the virtually concatenated channels will be multiplexed onto the individual channels (time slots) to be transported on the SONET network and then de-multiplexed at the other end.
The fact that multiple individual channels are transporting pieces of a common data stream places some requirements as to how the individual channels are created on the network. Specifically, since the multiple channels are actually time slots that each carry pieces of a common dataset, the delay between the multiple channels should be approximately the same so that the data does not arrive out of order at the far end of the network. If the delay on one of the channels is significantly longer than the delay of the other channels, the data from the other channels will need to be buffered while waiting for the data from the slower channel.
Accordingly, it is common to try to implement the channels so that they follow paths through the network that have similar delay characteristics, such that the delay between each of the virtually concatenated channels is approximately equal. While the connections do not need to go through the same path, to minimize the amount of delay difference between the channels ideally the channels should follow the same path and use the same physical links through the network.
Internet Engineering Task Force (IETF) Request For Comments (RFC) 4606 provides details about signaling VCAT on a SONET network using ReSerVation Protocol (RSVP), which enables MultiProtocol Label Switching (MPLS) Label Switched Paths (LSPs) to be set up across the SONET network so that multiple STS-1s (or STS-3c's) may be used to implement the concatenated connection. In GMPLS, Open Shortest Path First (OSPF) is commonly used as the routing protocol in the control plane. Nodes on the network advertise adjacencies with other nodes using link state advertisements. The nodes use the advertisements to build a topology database which may be used to route connections through the network. For scalability purposes, where a node has multiple physical links interconnecting itself with a neighboring node, the node will often summarize the multiple physical links into a single routing entry and advertise the link bundle, rather than advertising the individual links.
Links are only bundled where they have the same characteristics including propagation and delay characteristics. Thus, since VCAT enables different connections to be carried by different physical fibers as long as the delay characteristics are similar, it should be possible for different concatenated connections to use different fibers of a bundled link.
Unfortunately, RSVP, which is used to signal connections on GMPLS networks, does not support establishment of multiple VCAT related connections on different physical links between nodes, even if the links are part of a link bundle.
When RSVP is used to signal a connection on the network, it creates a Label Switched Path (LSP) through the network which is a finite state machine. The finite state machine controls the data plane of the network element to cause the data plane to establish the physical connectivity through the network. Succinctly, once the OSPF database has been consulted and a route for a connection has been determined, an RSVP “PATH” message is sent out on the network. The PATH message causes the resources to be reserved by the control plane of each node along the route through the network. Once the PATH message has reached the destination, a second RSVP message is sent back along the connection to cause the control plane to program the connection into the data plane at each node. This message is referred to as a “RESV” message. A node that generates the PATH message will be referred to herein as the “head-end node”.
Where the PATH message refers to multiple concatenated STS-1s (or STS-3c's), the nodes along the route will reserve multiple time slots on the same physical fiber. When the RESV message is received, the node will then program the data plane to implement the resource reservation. Thus, in this instance, a single Label Switched Path finite state machine (at each node) will control the concatenated reserved bandwidth for the connection at that node. Upon failure of the fiber or other data plane failure, the finite state machine can trigger the connection to failover to a new connection, since all of the VCAT connections are on the same fiber, the finite state machine can control failover of all of the data plane connections.
While the one finite state machine can control the multiple physical connections if the connections are all on the same fiber, the one finite state machine does not work well where the multiple connections are implemented on different physical fibers in the data plane. For example, the multiple different physical connections may go down at different times, etc., which makes it difficult for the one finite state machine to manage failover of multiple STS-1s (or STS-3c's) on multiple physical connections.
RSVP also can be used to separately signal each connection of a VCAT, which would enable the different connections to use different links of a link bundle. A problem arises, however, when the connections are required to traverse a network domain boundary where the route is required to be expanded. Typically, routing information for a domain is abstracted at network domain boundaries, which prevents the head-end node from calculating the entire route for the path through the network. While the head-end node will have routing information for its own domain, it will not have sufficient routing information for neighboring domains to calculate routes through those domains. Thus, when a PATH message is received at the ingress to a neighboring domain, the External Network to Network Interface (E-NNI) at the ingress to that domain will calculate a route for the connection through the neighboring domain. This is commonly referred to as route expansion. If PATH messages for separate connections of a VCAT are processed by the E-NNI separately, the E-NNI may perform route expansion differently for the related connections which means that there is no guarantee that the different connections will follow the same path through the neighboring domain. Accordingly, it would be advantageous to provide a way to ensure that multiple virtually concatenated channels are co-routed through the network even when route expansion is required to be performed on the connections.